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Abstract 

Student feedback surveys are an important component of the continuous improvement cycle at the 
University of Newcastle. This article describes a recently adopted holistic approach to collecting, 
analysing, reporting and acting on student feedback on courses (units of study), programs (collection 
of courses leading to an award) and the student experience. The approach demonstrates the value to 
staff and students of an integrated system of reporting, based on clear definitions of purpose, well 
designed data collection instruments and analysis, and an unambiguous and prompt feedback system 
to all participants. It is supported by ongoing consultation and conversations about the survey 
processes, outcomes, and ways of addressing the survey findings. 

A focus has been placed on follow-up support to assist academic staff and their supervisors to respond 
to findings. Reports summarising quantitative and qualitative data have been designed as well as 
communication guidelines for managers, developed in consultation with academic staff, and through 
ongoing dialogue with them, resulting in a more transparent and collaborative approach to measuring 
and improving quality. Survey instruments have been revised following extensive consultation and are 
now administered online. Reporting back to students has been improved with survey results and 
actions communicated in a multiplicity of ways. 

The development of this approach has seen a rise student engagement with surveys and escalating 
engagement with student feedback by academic staff. Students and staff are developing stronger links 
to their university communities as they both participate in and benefit from this more consultative and 
coordinated approach to collecting and acting on student feedback. 
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Data collection is one of the key stages in quality enhancement and this has long been 
a focus of evaluation systems in the higher education sector. However, the move from data 
collection to meaningful reports that assist with decision-making and targeted actions for 
improvement has proved a challenge for many institutions. This may be, in part, due to the 
perception that continuous improvement is more about accountability and assessment, rather 
than striving to add value by improving processes and systems (Lomas, 2004). Furthermore, 
as Houston (2008, p. 62) states, there is often a gap between the ‘rhetoric of quality and the 
practice of improvement’. 
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In higher education institutions, with their various subcultural norms and values, 
attempts to embed processes of critical reflection and actions towards continuous 
improvement without a holistic approach based on negotiated consensus-building, will often 
fail (Houston, 2008; Trowler, 2005). Indeed, improvement processes must take into account 
and reflect this divergence of needs and interests by developing accessible interventions, if an 
appreciation of quality enhancement principles is to become commonplace and influence 
practice. 

In recognition of this, the University of Newcastle has developed six principles to 
assist with embedding a culture of continuous improvement: consultation, communication, 
encouragement, celebration, integration, and innovation. These principles offer all 
stakeholders the opportunity to discover tangible and accessible activities that extend beyond 
the rhetoric of quality. This article will begin by outlining the role of data collection in the 
quality enhancement cycle, with mention of how failure to complete the rest of the cycle can 
lead to stakeholder backlash and negative outcomes. We will then explain our six principles 
and illustrate the holistic approach using our Student Feedback on Courses survey. We have 
made substantial changes to the type of data we collect, and how it is reported and 
communicated in a collegial and collaborative way to inform decisions and actions that 
enhance the student and staff experience. The example encapsulates how notions of quality 
have translated into tangible actions and manifested in improvements in student engagement 
and increased staff cooperation and collaboration across the university. 

Data Collection Within a Culture of Continuous Improvement 

Regular monitoring and review of activities are key processes towards the goal of 
embedding a culture of continuous improvement. The four quadrant quality enhancement 
cycle (see Figure 1) begins with development of a strategic plan that defines an organisation’s 
vision, purpose and goals. This, in turn, determines the nature of actions and information 
required, followed by implementation of a monitoring system, analysis and reporting of 
results, and recommendations for improvements that align with the overarching strategy. 

Traditionally, many institutions and businesses focus on the first half of this model, 
with the development and deployment of a strategy. However, while the literature shows the 
importance of the latter half of the quality enhancement cycle (see Houston, 2008; Lomas, 
2004; Trowler, 2005), there is little evidence in the higher education sector to demonstrate 
that institutions fully capitalise on the information once collected. 

Implicit in improvement is the necessity for change. Hence, motivation for 
completing the quality enhancement cycle has largely been contingent upon existing 
subcultural norms and the degree to which these are threatened by change, or indeed, 
consulted and challenged (Houston, 2008). Consequently, the quality enhancement cycle 
rarely comes to fruition. Instead, a system of high-level planning, communicating the plan, 
and collecting data and feedback tends to fragment as results are interpreted and filtered at 
the local subcultural level that often acts as ‘a brake on change’ (Houston, 2008, p. 75). 
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Figure 1 

The quality enhancement cycle. 


Engagement of Staff and Students With Quality Enhancement 

Integral to embedding a culture of continuous improvement in higher education 
institutions is the establishment of a total organisational focus where the motivation to 
continually improve processes and outcomes is derived via consensus and becomes 
normalised and commonplace. In this sense, an institution’s culture, or, ‘the way we do things 
around here’ (Deal & Kennedy, 1988, cited in Lomas, 2004, p. 159) needs to have quality 
embedded within underlying assumptions, beliefs, values, intentions, conversations, and 
processes, rather than coming from decree or focusing on outcomes and/or deficits alone 
(Houston, 2008; Lomas, 2004). 


In isolation, without closure to each evaluative process, data collection can be 
cumulatively detrimental to the culture of the organisation. When a student perceives that 
completion of a survey is the end of the process, there is a detrimental effect on sustained 
response rates, and may result in student disengagement from issues that are topical of the 
evaluation conducted, reinforcing any existing sense that the university does not care about 
them. Furthermore, negative feedback on teachers or courses can result in a culture of blame 
shifting, where disenfranchised staff members feel that change is beyond their scope or 
responsibility (Anderson, 2006; Moore & Kuol, 2005). 

A developed ‘local definition’ of quality (Carmichael et al., 2001 cited in Houston, 
2008) incorporating a consultative and inclusive process of determining the objectives of 
feedback, and then providing it to key stakeholders in a positive and affirming way, can assist 
in redressing combative reactions to entrenched subcultural perceptions of the purposes and 
outcomes of evaluation systems. 
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The University of Newcastle has recently experienced early success in implementing 
a culture of continuous improvement that has resulted in a positive shift in perceptions from 
the academic staff towards quality and hence greater cooperation. Our focus was largely 
centred on ‘consensus building as a pre-requisite to successful change’ (Houston, 2008). 

The University of Newcastle approach 

The holistic approach taken at the University of Newcastle includes a strong focus on 
the phases of communicating results and strategies to develop strategic activities and 
implement change. We are involved in assisting the teaching staff to develop effective 
practices for responding to feedback and communicating the responses to students. 

At the University of Newcastle we promote a culture of continuous improvement 
using six guiding principles: 

• Consultation—implies recognition of the needs of all stakeholders in the endeavour 
to achieve organisational goals. It involves coordination, engagement, collaboration, 
and continually informing and providing feedback about a process (Houston, 2008). 

• Communication—focused on the use of positive language, knowledge sharing, 
closing the feedback loop, and using best practice dissemination methods (Houston, 
2008; Johnson, 2003; McCormack, 2005; Morgan, 2008). 

• Encouragement—creation of an environment that supports: measured risk-taking, an 
individual’s ability to change, professional development, reflective practice, peer 
review, peer support, and mentoring practices (Houston, 2008; Lomas, 2004). It also 
encompasses targeted recruitment of staff whose values align with the quality culture 
paradigm. 

• Celebration—a focus on highlighting successes, sharing successful stories within 
communities of practice and providing genuine recognition of these efforts (Lomas, 
2004). 

• Integration—addressing quantitative bias (see Anderson, 2006) by supplementing 
statistical trends with qualitative data that identifies perceptions and experiences 
(Morgan, 2008). Integration also implies a total organisational focus on quality that 
reaches across faculties, units and services to engender cooperation and collaboration 
that transcends subcultural needs (Lomas, 2004). 

• Innovation—led by students’ identification of strengths and weaknesses through 
feedback mechanisms, such as surveys and focus groups (Lomas, 2004). Likewise, 
consultation with staff, as well as staff surveys and other forms of input, can further 
lead innovation. 

A culture of continuous improvement will also help engage staff in assessing progress against 
the key goals of our strategic plan ‘Building Distinction 2011-2015’. The first strategic 
priority of the strategic plan is promoting and improving a positive student experience and 
quality teaching and learning. The strategic plan has clear targets and measures for student 
satisfaction. We will continue to build strongly in this area, improving our capability to 
transform data into information and then communicate this for appropriate decision-making 
that meets improvement objectives. 

Our revised student feedback on courses process applies the principles of the quality 
culture system developed at the university and provides a case study that exemplifies the 
holistic approach taken to ensure coverage of all four quadrants of the quality enhancement 
model. 
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Continuous Improvement in Action: A Case Study 

A fundamental component of embedding a culture of continuous improvement is to 
recruit or assign staff to positions of responsibility for developing and implementing the 
improvement processes. This is also where we began to improve the way that student 
feedback was collected, analysed, reported and acted on (see Figure 2). 



Figure 2 

Student feedback on courses continuous improvement cycle. 

The university established the Planning, Quality and Reporting (PQR) unit in January 
2009. The recruitment strategies of this unit focused on employing staff who had a collegial, 
consultative style, experience in higher education and an understanding of the cultural 
challenges faced in that environment. The staff comprised a multidisciplinary team who 
quickly saw the potential for change and had the interpersonal skills to effectively promote 
and promulgate a way forward. One of the first moves was to put all student surveys online 
and to invite students to provide open-ended comments about aspects of their programs and 
courses that worked well and areas for improvement, in addition to the multiple-choice items 
that had been in use for several years. A project officer was employed in the team to code all 
comments and prepare reports at the course and program level for senior management to use 
in their faculties and schools. The integration of quantitative and qualitative results in a single 
report for academic staff—summarised to school, faculty and university level with 
comparative data—is not commonplace in the sector. More than 1500 individual Student 
Feedback on Courses reports were prepared during 2009 alone, incorporating 55,000 
comments that had been coded and summarised according to the broad themes of course 
design, staff, assessment, support and outcomes. 
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The PQR unit then assessed the results and drafted reports in consultation with 
academic staff and students. One of the early outcomes was the realisation that the survey 
instruments required updating to make the questions more meaningful to the students and the 
results more useful to staff. A new survey instrument was drafted and revised during ongoing 
consultation with staff and students, resulting in an instrument that allowed the university to 
better understand the students’ views on important aspects of their courses and assisted 
teaching staff to know where to make changes to their courses. 

The process involved successful cross-unit cooperation and direct interaction between 
students, academic staff, administrative support staff and senior leadership as well as PQR 
officers responsible for administering and analysing the survey and preparing reports. In 
order to develop an evaluative tool that integrated the expectations, needs, and outputs of all 
stakeholders (teaching staff, students, data analysts and managers), the instrument itself 
needed to be pilot tested. Course coordinators in courses delivered by four schools across two 
trimesters volunteered to be involved in the pilot testing phase. Students in these courses 
were aware that they were being asked to test the new instrument and were encouraged to 
provide comments on the length, the clarity and ease of completion of the survey. The email 
invitations to students to complete the survey were also pilot tested with students to ensure 
the language was upbeat and student friendly. 

Our focus during this time was on creating a culture of continuous improvement and 
ownership of the process, using positive language in our conversations and correspondence 
that was reflective of that used in the instrument, and providing affirming and encouraging 
feedback. The draft survey and reporting structure was sent to the Faculty and School 
Teaching and Learning Committees, Faculty Boards, Academic Senate and Executive 
Committee for feedback and eventual endorsement. 

Student feedback instruments have sometimes been criticised for not reflecting sound 
survey design and not being rigorously tested for validity and reliability (Anderson, 2006; 
Barrie & Ginns, 2007). The processes followed by the university adhere to good survey 
design and development principles. The components of the new Student Feedback on 
Courses forms have strong face validity and assess: expectations, support, learning activities, 
teaching, structure, organisation, resources, content, challenge, assessment, criteria, feedback, 
relevance, satisfaction, overall satisfaction, and a self-evaluation of effort made to succeed in 
the course. Students are invited to add comments at the end of each question and there are 
two final open-ended questions asking about best aspects of the course and areas for 
improvement. Further analysis of validity and reliability is being undertaken. 

The innovative approach taken to promote continuous improvement in our student 
feedback processes identified not only the need for a new instrument but also the need for 
delivery in a new medium. In consultation with staff from IT Services, a comprehensive 
software review was conducted in tandem with the instrument’s design. The aim was to 
implement a system that would support the depth, breadth and complexity of the new 
instrument and enable meaningful analysis and reporting, as well providing a seamless, user- 
friendly experience for students. 

Several software systems for delivering online surveys were investigated and 
assessed. The objectives of the project were to: 

• capture a more representative sample of students by administering surveys online 

• reduce the amount of manual handling and scanning required for paper surveys 
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• provide a confidential way for students to provide open-ended comments 

• enable students to use their usual login to access surveys 

• provide a single portal for each student to access their surveys 

• reduce the turnaround time for providing survey results to staff and students 

• ensure the survey software system was compatible with our current IT systems and 
knowledge base. 

The system called Blue from eXplorance™ (Canada) satisfied our selection criteria and has 
been implemented and tested in trimester 2 for roll out across all courses from semester 2 
2010 . 


Developing and encouraging a culture of continuous improvement requires 
willingness to think beyond the present and the way things have always been done, while 
building on the best aspects of organisational knowledge and wisdom of the past. The design 
and selection of both the survey instrument and the software to deliver it were the product of 
creatively addressing local needs as reported by students and staff and using sound research 
and business practices. The ongoing consultation between PQR and academic staff to monitor 
actions and progress is a critical element of the approach. 

From Data to Information to Improvement 

Senior management require reports that assist with decision-making at a strategic 
level. In consultation with the senior leadership group, the PQR unit designed several 
university-wide reports showing results across all faculties and schools, areas of excellent 
practice and areas requiring focused support. Several additional pieces of information, such 
as class size, attrition and grade point average have been incorporated into the reports so that 
decisions can be made about where best to focus resources to achieve maximum gain in 
improving teaching and learning quality and the student experience. Once again, the language 
of these reports is reflective of a quality culture that supports and encourages improvement. 

Increasing the awareness of quality enhancement among staff and students, 
particularly regarding the quality of teaching and learning, not only invests measurability in 
achieving improvement but also adds recognition and support in less quantifiable but equally 
important ways, and ‘in higher education, [it] places student learning “at the heart of quality’” 
(Carmichael et ah, 2001 in Houston, 2008, p. 69.). 

Communication guidelines have been developed to assist heads of school to act on the 
survey findings and communicate findings and resultant actions to their staff and students. 

For example, heads of school are encouraged to prepare summaries identifying recurring 
themes across the school or discipline, to encourage faculty-wide discussion and 
collaboration and, where appropriate, to enable an integrated and holistic approach to 
addressing issues and sharing exemplary practices. Where there are sensitive issues that need 
addressing, the guidelines suggest that heads of school organise a discrete conversation with 
the appropriate staff first, before any course-specific comments are distributed. This will 
enable the talking through of issues, in a calm and safe manner, and facilitate assessment of 
their validity or otherwise. Students can be informed of survey results in a multiplicity of 
ways through the web, Blackboard®, email, posters, in course outlines and by sending 
summaries of actions to student representatives to give them the opportunity to feedback to 
the student groups and associations. 
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Of further import, a culture of inclusive and holistic reporting of results has been 
undertaken using positive language that celebrates success and contextualises criticism in a 
constructive framework. This obviates any need for blame, but rather creates a supportive 
base for change. Using templates designed by the PQR unit for reporting on actions, dialogue 
has started with academic teaching staff, course coordinators and heads of schools about 
identifying areas for positive action. These tools reinforce the concept of supported change 
rather than the negatively framed ‘areas of deficit for redress’. 

Through faculty teaching and learning committees, the courses with overall 
satisfaction scores in the lowest 20% and the programs with scores in the lowest quartile have 
been selected for attention. Course coordinators and program convenors are working with 
PQR staff to identify why the courses and programs scored less well (informed by the 
qualitative data) and what actions can reasonably be taken to have the best impact on 
improving the student learning experience. Reports on actions and outcomes are prepared for 
the Teaching and Learning Committee and reports are available at a university level to allow 
sharing of good practice. 

Student feedback reporting at program and course level facilitates conversations that 
are relevant at the local level and encourages the culture shift. When feedback on teaching is 
incorporated as a tool for reflective practice and targeted professional development, rather 
than as a ‘witch-hunt’, a culture of continuous improvement is more likely to develop. The 
new data collection, reporting and dissemination process not only allows, but requires 
celebration of those courses that have rated highly, and the noting of the positive aspects most 
commented on by students. It is for the benefit of all to celebrate the successes and adopt 
good models across discipline boundaries. 

The holistic approach includes an ongoing schedule of face-to-face meetings between 
PQR and academic teaching staff and heads of schools. These conversations celebrate the 
great work achieved and provide an environment that is encouraging and affirming and 
narrows the perceived gap between academic and administrative areas of the university. 

Currently, the suite of evaluative processes at the university includes surveys, focus 
groups and student forums. These processes seek students’ experiences at program and 
course level, including their perceptions of the quality of teaching, the range of services 
available to them and how accessible and useful the services are. Our quality enhancement 
initiatives have included a comprehensive revision of our policies and procedures for our 
external review of programs and our annual reporting on programs. More meaningful and 
actionable results from these revised quality enhancement processes have led to collaborative 
workshops, such as the Student Experience Workshop, where senior academic and support 
staff from areas across the university discussed issues, proposed solutions and developed 
action plans with measurable outcomes and targets. 

Conclusion 

The University of Newcastle’s growing awareness and implementation of a culture of 
continuous improvement is exemplified in the redesign of our student feedback on courses 
procedure. Through a process of consultation, communication, encouragement, celebration, 
integration and innovation we have been able to complete the quality enhancement cycle and 
achieve improvements. Along the way we have strengthened relationships, cross-institutional 
collaboration and communication, and ultimately embedded notions of quality into 
underlying assumptions, values, beliefs, conversations and actions. Our principles recognise 
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and address subcultural needs and interests while also bridging the divide between quality 
rhetoric and quality implementation. Overall, our approach has seen improvement in 
processes and practices, which has ultimately led to increasing student engagement with the 
student feedback process. Students report that they now believe that someone is actually 
listening to them and acting on their feedback. Senior managers now have the tools and 
information they need to lead change. 

The development of this approach has seen an increase in the coverage and response 
to internal, national and international surveys and escalating engagement in the value of the 
feedback process by academic staff. Students and staff are developing stronger links to their 
university communities as they both participate in and benefit from this more consultative 
and coordinated approach to collecting and acting on student feedback. 
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